Hi...I have a task where I need to transform the data from source to destination. The source table has about 40 million rows and keep growing rapidly. At present I am truncating the destination table everyday and reloading the data every day which is a time consuming process. I am not able to find any logic to implement the append logic so that I only tranfer the data from last time stamp when the job runs. The source table has four columns with no unique key identifier(all varchar) and and timestamp column. Since this is a power metering data, the timestamp is the actual meter reading time. The data in the source is coming from the various time zones. I can not use the timestamp value as the unique identifier...for example if I ran the job today at 12 am, my destination table had the max timestamp row as 12.15 AM. The next day when I run the job I can not use the condition to get the data from 12.15 onwards from source since there might have some data in the source of before 12.15 am which is not in the destination yet (from the timezones where the time is lagging). Please let me know if there is an ulternate way available for this? Thanks, Gaurav

Sounds like you're screwed. Any chance of modifying the source table? Add a [DateTime Appended] field with a default value of GETDATE() ?Todd C - MSCTS SQL Server 2005 - Please mark posts as answered where appropriate.

Todd's right. If you (as a human) can't figure out a way to detect altered rows, SSIS can't do it either. Talk to me now on

thans Todd C and Todd McDermid...I think you are right. I think the only way to make this working is adding the DateTime (GetDate) in the source table.Thanks, Gaurav

Work around: either add another column to your table as you mention above but I would suggest you to add that in source as well as in destination. Let us TRY this | My Blog :: http://quest4gen.blogspot.com/

I would suggest you to add that in source as well as in destination. Oh, abso-frickin'-lutely! Maybe a slightly better solutin instead of the DateAppended column, would be a [SourceSystem] column, which would tie back to a table of data providers which would have in it some information about the Time Zone that the Meter Reader is in. That way, one could determine the UTC timestamp of the record. Either way, As "ETL" suggests, anything you glean from your Source system should probably be included in the Destination.Todd C - MSCTS SQL Server 2005 - Please mark posts as answered where appropriate.